Allen County
Y'all versus yinz: Accents may say more about who we are than where we're from
Science Y'all versus yinz: Accents may say more about who we are than where we're from There's more to our speech than meets the ear. More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. Influences on accents go far beyond regional upbringing. Breakthroughs, discoveries, and DIY tips sent six days a week. Regional accents in the United States are far more complicated than their oversimplified stereotypes .
Learning Overspecified Gaussian Mixtures Exponentially Fast with the EM Algorithm
Assylbekov, Zhenisbek, Legg, Alan, Pak, Artur
We investigate the convergence properties of the EM algorithm when applied to overspecified Gaussian mixture models -- that is, when the number of components in the fitted model exceeds that of the true underlying distribution. Focusing on a structured configuration where the component means are positioned at the vertices of a regular simplex and the mixture weights satisfy a non-degeneracy condition, we demonstrate that the population EM algorithm converges exponentially fast in terms of the Kullback-Leibler (KL) distance. Our analysis leverages the strong convexity of the negative log-likelihood function in a neighborhood around the optimum and utilizes the Polyak-ลojasiewicz inequality to establish that an $ฮต$-accurate approximation is achievable in $O(\log(1/ฮต))$ iterations. Furthermore, we extend these results to a finite-sample setting by deriving explicit statistical convergence guarantees. Numerical experiments on synthetic datasets corroborate our theoretical findings, highlighting the dramatic acceleration in convergence compared to conventional sublinear rates. This work not only deepens the understanding of EM's behavior in overspecified settings but also offers practical insights into initialization strategies and model design for high-dimensional clustering and density estimation tasks.
Self-supervised Topic Taxonomy Discovery in the Box Embedding Space
Lu, Yuyin, Chen, Hegang, Mao, Pengbo, Rao, Yanghui, Xie, Haoran, Wang, Fu Lee, Li, Qing
Topic taxonomy discovery aims at uncovering topics of different abstraction levels and constructing hierarchical relations between them. Unfortunately, most of prior work can hardly model semantic scopes of words and topics by holding the Euclidean embedding space assumption. What's worse, they infer asymmetric hierarchical relations by symmetric distances between topic embeddings. As a result, existing methods suffer from problems of low-quality topics at high abstraction levels and inaccurate hierarchical relations. To alleviate these problems, this paper develops a Box embedding-based Topic Model (BoxTM) that maps words and topics into the box embedding space, where the asymmetric metric is defined to properly infer hierarchical relations among topics. Additionally, our BoxTM explicitly infers upper-level topics based on correlation between specific topics through recursive clustering on topic boxes. Finally, extensive experiments validate high-quality of the topic taxonomy learned by BoxTM.
SNNLP: Energy-Efficient Natural Language Processing Using Spiking Neural Networks
Knipper, R. Alexander, Mishty, Kaniz, Sadi, Mehdi, Santu, Shubhra Kanti Karmaker
As spiking neural networks receive more attention, we look toward applications of this computing paradigm in fields other than computer vision and signal processing. One major field, underexplored in the neuromorphic setting, is Natural Language Processing (NLP), where most state-of-the-art solutions still heavily rely on resource-consuming and power-hungry traditional deep learning architectures. Therefore, it is compelling to design NLP models for neuromorphic architectures due to their low energy requirements, with the additional benefit of a more human-brain-like operating model for processing information. However, one of the biggest issues with bringing NLP to the neuromorphic setting is in properly encoding text into a spike train so that it can be seamlessly handled by both current and future SNN architectures. In this paper, we compare various methods of encoding text as spikes and assess each method's performance in an associated SNN on a downstream NLP task, namely, sentiment analysis. Furthermore, we go on to propose a new method of encoding text as spikes that outperforms a widely-used rate-coding technique, Poisson rate-coding, by around 13\% on our benchmark NLP tasks. Subsequently, we demonstrate the energy efficiency of SNNs implemented in hardware for the sentiment analysis task compared to traditional deep neural networks, observing an energy efficiency increase of more than 32x during inference and 60x during training while incurring the expected energy-performance tradeoff.
Intractability of Learning the Discrete Logarithm with Gradient-Based Methods
Takhanov, Rustem, Tezekbayev, Maxat, Pak, Artur, Bolatov, Arman, Kadyrsizova, Zhibek, Assylbekov, Zhenisbek
The discrete logarithm problem is a fundamental challenge in number theory with significant implications for cryptographic protocols. In this paper, we investigate the limitations of gradient-based methods for learning the parity bit of the discrete logarithm in finite cyclic groups of prime order. Our main result, supported by theoretical analysis and empirical verification, reveals the concentration of the gradient of the loss function around a fixed point, independent of the logarithm's base used. This concentration property leads to a restricted ability to learn the parity bit efficiently using gradient-based methods, irrespective of the complexity of the network architecture being trained. Our proof relies on Boas-Bellman inequality in inner product spaces and it involves establishing approximate orthogonality of discrete logarithm's parity bit functions through the spectral norm of certain matrices. Empirical experiments using a neural network-based approach further verify the limitations of gradient-based learning, demonstrating the decreasing success rate in predicting the parity bit as the group order increases.
Long-Tail Theory under Gaussian Mixtures
Bolatov, Arman, Tezekbayev, Maxat, Melnykov, Igor, Pak, Artur, Nikoulina, Vassilina, Assylbekov, Zhenisbek
We suggest a simple Gaussian mixture model for data generation that complies with Feldman's long tail theory (2020). We demonstrate that a linear classifier cannot decrease the generalization error below a certain level in the proposed model, whereas a nonlinear classifier with a memorization capacity can. This confirms that for long-tailed distributions, rare training examples must be considered for optimal generalization to new data. Finally, we show that the performance gap between linear and nonlinear models can be lessened as the tail becomes shorter in the subpopulation frequency distribution, as confirmed by experiments on synthetic and real data.
Can AI answer your money questions? We put chatbots to the test
NEW YORK, April 13 (Reuters) - Face it, we could all use a little help with our money. So who better to ask for personal finance advice than a couple of the most powerful chatbots on the planet? Both OpenAI's ChatGPT and Google's Bard are dominating headlines recently, for their generative capabilities and vast storehouses of information. Each has far more processing power than, say, any individual personal finance writer (ahem). What is one great business idea?
Senior Communications Software Engineer - Fort Wayne, IN Job in Fort Wayne, IN
Secmation is an equal opportunity employer. Qualified Applicants will be considered without regard to age, race, creed, color, national origin, ancestry, marital status, sex, affectional or sexual orientation, gender identity or expression, disability, nationality, or veteran status. Certain positions at Secmation may require specific physical/mental abilities. Additional information and provisions for reasonable accommodation will be provided by the hiring manager.